1. Field of the Invention
The present invention relates to an image capturing apparatus, an image processing method and program. More specifically the invention relates to an image capturing apparatus, an image processing method and program to provide from a group of images captured by a camera array, an image free from camera shake.
2. Description of the Related Art
As image capturing apparatuses have been decreasing in size and cost in recent years, camera arrays (as known as camera array system, multiple lens camera, and the like) are being proposed that can realize a zooming function by capturing images of a subject using a plurality of lenses with different focal distances without the use of an optical zoom (e.g., Japanese Patent Laid-Open No. 2005-109623). In a camera array image capturing apparatus proposed by Japanese Patent Laid-Open No. 2005-109623, an angle of view is made variable by selecting an image acquired by a lens with an optimal focal distance that matches a desired zoom ratio. This allows a single image capture apparatus to capture images in a wide range of view angle, without requiring a drive mechanism for zooming.
Recent years have seen a growing trend for a higher resolution and a higher zoom ratio of image capturing apparatuses, which has led to a problem of blurs in captured images caused by camera shake during an image capturing operation. Under these circumstances, cameras with a camera shake correcting function have come into wide use. Among such camera shake correcting methods there is an optical camera shake correction method that optically corrects camera shake by using a gyro sensor for detecting a camera shake angular velocity and a drive apparatus that controls a relative position between an image capturing lens and an image capturing element so as to cancel the camera shake. An electronic camera shake correcting method is also being proposed which generates an image with reduced camera shake by first capturing a plurality of images of a subject with a fast shutter speed and then electronically align and overlap the plurality of images in position (e.g., Japanese Patent Laid-Open No. 2006-074693). The electronic camera shake correcting method as proposed by Japanese Patent Laid-Open No. 2006-074693 does not require the gyro sensor and therefore can reduce the size of a camera.
Further, a technique called a Coded Exposure is also being proposed which performs a deconvolution calculation using shutter open/close information to correct camera shake or the shake of a subject itself (e.g., U.S. Patent Publication No. 2007/0258706). More specifically, this method calculates a distribution of shakes that occur during a period of capturing images by using a gyro sensor or the like and, based on an exposure time and the shake distribution, estimates a point spread function (PSF), a blur function of a point image on an interested image. This method further calculates a frequency characteristic of PSF and filters the image through a filter with a reversed frequency characteristic, or Wiener filter, to achieve an appropriate camera shake correction.
With the method proposed by Japanese Patent Laid-Open No. 2005-109623, however, since it does not employ a mechanism to correct camera shake, when an image of a subject is captured with a lens with a long focal distance in particular, blurs caused by camera shake become more likely to appear in the captured image. However, if an optical camera shake correction mechanism is mounted on each lens to avoid this problem, a lens unit will increase in size. In camera array image capturing with a plurality of lenses, this makes the overall size of the camera prohibitively large.
The method proposed by Japanese Patent Laid-Open No. 2006-074693 requires a plurality of images to be taken in, which means it is necessary to provide enough memory to accommodate the plurality of images for a position alignment operation. If, however, such an electronic camera shake correcting method is applied to all image capturing units of a camera array image capturing apparatus proposed by Japanese Patent Laid-Open No. 2005-109623, a large amount of memory is required, pushing up the manufacturing cost.
With the method proposed by U.S. Patent Publication No. 2007/0258706, the shutter needs to be operated randomly during exposure and at the same time an additional mechanism such as a gyro sensor to detect camera shake during an image capturing operation must be provided. If the shutter control during exposure is applied to all image capturing units of a camera array image capturing apparatus proposed by Japanese Patent Laid-Open No. 2005-109623, the mechanism for shutter control becomes complicated.